lie detection
Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs
Fu, Yao, Li, Runchao, Long, Xianxuan, Yu, Haotian, Han, Xiaotian, Yin, Yu, Li, Pan
Neural network pruning has emerged as a promising approach for deploying LLMs in low-resource scenarios while preserving downstream task performance. However, for the first time, we reveal that such pruning disrupts LLMs' internal activation features crucial for lie detection, where probing classifiers (typically small logistic regression models) trained on these features assess the truthfulness of LLM-generated statements. This discovery raises a crucial open question: how can we prune LLMs without sacrificing these critical lie detection capabilities? Our investigation further reveals that naively adjusting layer-wise pruning sparsity based on importance inadvertently removes crucial weights, failing to improve lie detection performance despite its reliance on the most crucial LLM layer. To address this issue, we propose Truthful Pruning aligned by Layer-wise Outliers (TPLO), which places greater emphasis on layers with more activation outliers and stronger discriminative features simultaneously. This preserves LLMs' original performance while retaining critical features of inner states needed for robust lie detection. Moreover, we introduce a prompting rule to enrich the TruthfulQA benchmark for better calibrating LLM pruning. Empirical results show that our approach improves the hallucination detection for pruned LLMs (achieving 88% accuracy at 50% sparsity) and enhances their performance on TruthfulQA.
- North America > United States (0.46)
- Europe > Spain > Catalonia > Barcelona Province > Barcelona (0.04)
- Europe > Italy (0.04)
- (2 more...)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning > Regression (0.86)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.71)
Bi-GRU Based Deception Detection using EEG Signals
Avola, Danilo, Bilal, Muhammad Yasir, Emam, Emad, Lakasz, Cristina, Pannone, Daniele, Ranaldi, Amedeo
Deception detection is a significant challenge in fields such as security, psychology, and forensics. This study presents a deep learning approach for classifying deceptive and truthful behavior using ElectroEncephaloGram (EEG) signals from the Bag-of-Lies dataset, a multimodal corpus designed for naturalistic, casual deception scenarios. A Bidirectional Gated Recurrent Unit (Bi-GRU) neural network was trained to perform binary classification of EEG samples. The model achieved a test accuracy of 97\%, along with high precision, recall, and F1-scores across both classes. These results demonstrate the effectiveness of using bidirectional temporal modeling for EEG-based deception detection and suggest potential for real-time applications and future exploration of advanced neural architectures.
Enhancing Lie Detection Accuracy: A Comparative Study of Classic ML, CNN, and GCN Models using Audio-Visual Features
Abdelwahab, Abdelrahman, Vishnubhatla, Akshaj, Vaswani, Ayaan, Bharathulwar, Advait, Kommaraju, Arnav
Inaccuracies in polygraph tests often lead to wrongful convictions, false information, and bias, all of which have significant consequences for both legal and political systems. Recently, analyzing facial micro-expressions has emerged as a method for detecting deception; however, current models have not reached high accuracy and generalizability. The purpose of this study is to aid in remedying these problems. The unique multimodal transformer architecture used in this study improves upon previous approaches by using auditory inputs, visual facial micro-expressions, and manually transcribed gesture annotations, moving closer to a reliable non-invasive lie detection model. Visual and auditory features were extracted using the Vision Transformer and OpenSmile models respectively, which were then concatenated with the transcriptions of participants micro-expressions and gestures. Various models were trained for the classification of lies and truths using these processed and concatenated features. The CNN Conv1D multimodal model achieved an average accuracy of 95.4%. However, further research is still required to create higher-quality datasets and even more generalized models for more diverse applications.
- North America > United States > Michigan (0.05)
- Europe > Germany > Bavaria > Upper Bavaria > Munich (0.04)
- North America > United States > Wisconsin (0.04)
- (4 more...)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (0.94)
- Health & Medicine (0.93)
- Law (0.66)
- Information Technology > Artificial Intelligence > Vision (1.00)
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (0.95)
How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions
Pacchiardi, Lorenzo, Chan, Alex J., Mindermann, Sören, Moscovitz, Ilan, Pan, Alexa Y., Gal, Yarin, Evans, Owain, Brauner, Jan
Large language models (LLMs) can "lie", which we define as outputting false statements despite "knowing" the truth in a demonstrable sense. LLMs might "lie", for example, when instructed to output misinformation. Here, we develop a simple lie detector that requires neither access to the LLM's activations (black-box) nor ground-truth knowledge of the fact in question. The detector works by asking a predefined set of unrelated follow-up questions after a suspected lie, and feeding the LLM's yes/no answers into a logistic regression classifier. Despite its simplicity, this lie detector is highly accurate and surprisingly general. When trained on examples from a single setting -- prompting GPT-3.5 to lie about factual questions -- the detector generalises out-of-distribution to (1) other LLM architectures, (2) LLMs fine-tuned to lie, (3) sycophantic lies, and (4) lies emerging in real-life scenarios such as sales. These results indicate that LLMs have distinctive lie-related behavioural patterns, consistent across architectures and contexts, which could enable general-purpose lie detection.
Interpretability's Alignment-Solving Potential: Analysis of 7 Scenarios - LessWrong
In each of the scenarios below, I'll discuss specific impacts we can expect from that scenario. In these impact sections, I'll discuss general impacts on the four components of alignment presented above. I also consider more in depth how each of these scenarios impacts several specific robustness and alignment techniques. To help keep the main text of this post from becoming too lengthy, I have placed this analysis in Appendix 1: Analysis of scenario impacts on specific robustness and alignment techniques. I link to the relevant parts of this appendix analysis throughout the main scenarios analysis below.
A Mental Trespass? Unveiling Truth, Exposing Thoughts and Threatening Civil Liberties with Non-Invasive AI Lie Detection
Sen, Taylan, Haut, Kurtis, Lomakin, Denis, Hoque, Ehsan
Imagine an app on your phone or computer that can tell if you are being dishonest, just by processing affective features of your facial expressions, body movements, and voice. People could ask about your political preferences, your sexual orientation, and immediately determine which of your responses are honest and which are not. In this paper we argue why artificial intelligence-based, non-invasive lie detection technologies are likely to experience a rapid advancement in the coming years, and that it would be irresponsible to wait any longer before discussing its implications. Legal and popular perspectives are reviewed to evaluate the potential for these technologies to cause societal harm. To understand the perspective of a reasonable person, we conducted a survey of 129 individuals, and identified consent and accuracy as the major factors in their decision-making process regarding the use of these technologies. In our analysis, we distinguish two types of lie detection technology, accurate truth metering and accurate thought exposing. We generally find that truth metering is already largely within the scope of existing US federal and state laws, albeit with some notable exceptions. In contrast, we find that current regulation of thought exposing technologies is ambiguous and inadequate to safeguard civil liberties. In order to rectify these shortcomings, we introduce the legal concept of mental trespass and use this concept as the basis for proposed regulation.
- North America > United States > California (0.05)
- North America > United States > Illinois (0.04)
- North America > United States > Oregon (0.04)
- (4 more...)
- Research Report (0.64)
- Overview (0.48)
- Questionnaire & Opinion Survey (0.46)
- Law > Statutes (1.00)
- Law > Civil Rights & Constitutional Law (1.00)
- Information Technology > Security & Privacy (1.00)
- (3 more...)
The race to create a perfect lie detector – and the dangers of succeeding
We learn to lie as children, between the ages of two and five. By adulthood, we are prolific. We lie to our employers, our partners and, most of all, one study has found, to our mothers. The average person hears up to 200 lies a day, according to research by Jerry Jellison, a psychologist at the University of Southern California. The majority of the lies we tell are "white", the inconsequential niceties – "I love your dress!" – that grease the wheels of human interaction. But most people tell one or two "big" lies a day, says Richard Wiseman, a psychologist at the University of Hertfordshire. We lie to promote ourselves, protect ourselves and to hurt or avoid hurting others. The mystery is how we keep getting away with it. Our bodies expose us in every way. We stutter, stall and make Freudian slips. "No mortal can keep a secret," wrote the psychoanalyst in 1905.
- Europe > United Kingdom > England > Hertfordshire (0.24)
- Asia > Afghanistan (0.14)
- Asia > India (0.14)
- (17 more...)
Will augmented reality make lying obsolete?
The most underappreciated application for the combination of augmented reality (A.R.) and artificial intelligence (A.I.) is persistent lie detection. Smartphones and smart glasses will soon support apps that show you in real time whether the person you're talking to is telling the truth or lying. Imagine how that will affect business meetings, sales presentations, job interviews and department status updates. Some 35 years ago, late-night talk show host Johnny Carson imagined what it would be like if politicians were hooked up to lie detectors.) Soon, you won't have to imagine it.
- North America > United States > Utah (0.05)
- North America > United States > California (0.05)
- North America > United States > Arizona (0.05)
- Asia > Middle East > Israel (0.05)
Are you lying about your identity? Artificial intelligence can tell by how you use your mouse
By tracking cursor movement, lie detection becomes a game of cat and mouse. Every year, millions of people have their identities stolen. There's no foolproof way to pinpoint fakers, but thanks to Italian researchers, investigators may soon have another tool at their disposal--a way to suss out frauds and other liars online with just a few clicks of a mouse. Traditional methods of lie detection include face-to-face interviews and polygraphs that measure heart rate and skin conductance. But they can't be done remotely, or with large numbers of people.
- Europe > United Kingdom (0.05)
- Europe > Netherlands > North Holland > Amsterdam (0.05)
- Europe > Italy (0.05)
Microsoft Ignite NZ 2016: Jennifer Marsman on machine learning, lie detection, and women in tech
Apple CEO Tim Cook says you won't have to give up your privacy to have a great AI assistant Tim Cook on A.I.: "I Don't Think We Have to Throw Our Privacy Away" Now AI is Deliberately Trying to Scare Us, if We Aren't Already TIM COOK: Here's why assistants on phones are better than home speakers like the Echo Stay up-to-date on the topics you care about. We'll send you an email alert whenever a news article matches your alert term. It's free, and you can add new alerts at any time.